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1 . Introduct ion 



I Background for. the Study 




Today f the role of rationality in organization, in 4eci- 
s ion-malcing, and in the accumulation of knowledge is of* paramount 
importance. Industrialization was the prime mover in elevating 
rationality to such exulted heights. Modern industrial societies 
constantly are attempting to apply rational criteria to more and 
more spheres of daily life. In our modem industrial societies, 
research has increasingly become the major intellectual method for 
reaching rational decisions. Accordingly, such societies devote 
ever greater • amounts of time, personnel, and resources for research 
activities. Indeed, a hallmark of these societies is the support 
and faith they display towards research, both pure end applied. No- 
where is this devotion and encouragement of research more evident 
than it is here in the United' States. The funds spent by £he govern- 
ment, foundations, and private corporations on research in the United 
States annually run into the billions. Furthermore, . the amount of 
time devoted to research activities by faculty, staff, and students 
at American universities is enormous. 
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Modern scientific research was initiated in the physical 
and biological sciences , and its application to agricultural prob- 
lems gave rise to the development of modern statistics . In' the last 
several decades, however, complex research designs have made their 
appearance in many other fields. In the United States, a dramatic 
mushrooming of research activities in the social and behavioral 
sciences, as well- as in such related fields as* education, welfare, 
and administration, occurred during the post war era. Funds sudden- 
ly. became available to research many aspects of social life, psycho- 
logical phenomena, economic development, and political behavior. 

In response to this unprecedented demand for research in 7 
these areas of the social and behavioral sciences, undergraduate and 
graduate curriculums have expanded their irifethodology offerings*. In > 
sociology, for example, undergraduate majors at the largerAijierican 
universities are usually required to take at the minimum a one semes- 
ter course .in statistics and a\one semester course in methodology. 

The minimi! requirement for graduate students in sociology at mLiy 
departments is two semesters each of statistics and methodology A In 
. addition, .such graduate students are .often encouraged to enroll in. 
one or more advanced courses focusing on survey methods, demographic 
analysis, field work or experimental design. In general, the method- 
ology courses offered to both undergraduate and graduate students at\ 
American universities stress quantitative modes of. analysis. In the] 
fields of sociology and political science these methodology offerings 
focus on survey methods, while in psychology and education experiment- 
al methods predominate. In recent years, however, survey technique's l 
have; inroads .in numerous additional fields, including educational \ 
research. Accordingly, an Increasing number of methodology courses 
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offered at American universities devote at least part of their time [ 
to a< discussion of survey research techniques. * 

x ’ i _ • < 

Paralleling this expansion of methodology offerings in the 
social and behavioral sciences has been a corresponding expansion in 
the number of textbooks designed to be used in methodology courses. 

Mopt of these texts ajre geared to advanced undergraduates or first 
year graduate students and their scope is both broad and general. 

The topics discussed in such texts include the following: • Philosophy 
of science, research design, procedures for data collection arid pro- 
cessing, techniques preparatory to or concomitant with data analyst, 
f data analysis itself, formats for the presentation of findings, and 
comments concerning the role of research. These texts cover survey 
and experimental methods, though in a brief and superficial manner. 

In addition to these general methodology textbooks., thdre are a few 
specialized texts intended for graduate level work that focus in 
depth on survey techniques . — * • * ■ 

* The increasing exposure of social science students to 
methodology courses in which survey 'methods are covered and the rapid 
multiplication of texts available to students enrolled in such class- 
es ,. imply that social* science students are being adequately trained • 
to carry out survey research. Unfortunately, this is not so. Both 
the procedures utilized in the classroom by instructors teaching survey 
research methods and the contents of most methodology texts that dis- 
cuss shrvey techniques inadequately prepare students for the task of 
analyzing' survey data. While some methods of research, such as exper- 
imental., clearly anticipate the detailed steps of analysis; other 
methods, such as survey, frequently fail during the design stage to 
foresee the analytical avenues through which the researcher will 
^travel. Consequently, although a primary focus on the problems* and ? 
techniques of research design is appropriate for training in exper- 
imental methods, it is insufficient when training students in the 
skills of survey analysis. 



, After the survey questionnaire has been designed, the 
interviews scheduled, responses recorded,! and the data transferred 
onto IBM cards or computer tape, the real work of the survey research- 
.er begins. He is often inundated with a large amount of data,* and 
It is difficult to tell the "trees from, the forest." Launching an 
analysis of survey . data, which contain tens of thousands of tables 
or correlational coefficients, is like finding one's way out of an 
elaborate, confusing labyrinth. Many students never succeed in work- 
ing their way out of such a maze. The survey analyst is faced with 
the task of finding* a > path through the - labyrinth to reveal a coherent 
explanation of the phenomenon under investigation. The emphasis is 
•thus on developing skills in discovery rather, than on methqds of 
verification . Unfortunately, students at Americah universities 
today are often" confronted with classroom instruction and textbooks 
which focus- only on methods of verification and fail to develop 
competence in their ability to discover relationships and to link 
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a serf.es of relatively simple relationships into a complex, coherent 
structure. This project was en attempt to explore ways of filling' 
this void . . ’ • ;• * 

l 4 1 # • 

f ^ # 

2< . Limitations of methodology texts * 

\ * 

Despite the increase in the number and variety of method- 
ology textbooks being turned out by American' publishers , it is our 
, contention that they, fail to devote sufficient space to a discussion • 

: of how to propeed in analyzing survey data. This impression is 

borne out by a crude content analysis that we carried out of the 
material presented in approximately forty textbooks which wefe in- 
tended for use in first year methodology courses in sociology. 

, Almost one-third of the pages of these textbooks were devoted to a 
discussion of the problems, procedures, and techniques which precede' 
the collection of data -- e.g. , developing hypothesis', research 
- design, questionnaire construction, and sampling. This comprised 

the largest amount of space devoted to any one particular phase of 
\ the research enterprise. Another one-quartP.r of the pages of these 

textbooks was devoted to the problems, procedures, and ^techniques 
of analysis and presentation of the collected data -- e t g., general 
problems of measurement; construction of indices, scales, and 
factors; development of coding categories ; statistical techniques; 
and the use of graphs, tables, and diagrams to summarize the present- 
ation of research findings . While a mastery of these skills is a 
necessary prerequisite to becoming a competent survey analyst, in 
and of themselves, they are insufficient in aiding a student to 
^ launch his , own analysis of survey data. They do not take him very 
far in his quest for a path out of the labyrinth pf rich data with 
which he’ is confronted. * 
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Approximately one- seventh of the pages of the texts * 

surveyed foqused on such questions as the relationship of research 
to theory, issues 'in the philosophy of science, and the implica- 
tions and broader context of; research. Another one -seventh of • 
these pages was taken up .with, a discussion of- how data is actually 
gathered- — e.g., interviewing Arid observational j techniques . 

Finally, less than one-tenth 1 of the pages in these tekts was devot- 
ed directly to a discussion of how to analyze data. • Furthermore, 
the presentation was rarely directed exclusively towards' the most , 
frequently used research method in American sociology — njpamely, 
survey analysis. Even when the focus was on survey analysis, the 
discussion was frequently i limited to explanation of how to read 
two and three variable tables , 'rather than on how ito discover 
relationships , and how to. link them together into a comprehensive 
analysis . * , \ \ 

/ f ■ * • . 

* f Up to this 5 , point, we have limited our. discussion to gen- 
eral methodology textbooks designed for first year methods courses in 
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sociology. What about the jsore specialized methodology texts in- * 
tended for graduate level courses and which exclusively deal with ‘ 
survey research? Do not such texts as Hyman’s Survey Design and 
Analysis , Zeisel's Say It With Figures , and Rosenberg's The Logic 
of Survey Analysis adequately prepare the student for undertaking 
his. own analysis oi survey data? It is true that such texts do 
devote a considerable amount of space to a discussion of the actual 
steps' involved in the analysis of survey data.' Even these texts, 

' however, emphasize the individual' steps of data analysis in isola- 
t ion to each other rather than in linking ^hem together to form a 
comprehensive and smoothly flowing analysis. Thus \ for example, 
|these text s^- usually focus. on how to read, analyze, and interpret 
individual two and three variable tables, as well as how to 
"elaborate" a simple- two variable relationship by the systematic 
introduction of different third "test" variables. Little consider- 
ation, however, is given to developing the student's skill in link- 
ing a number of these tables together in order to form a comprehen- 
sive, relatively complex explanatory structure. This is a serious 
omission on the part of these texts since much survey research is- 
initiated with the hope of coming up with such an explanatory / > 

structure for practical or theoretical purposes. 
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I We will now shift our attention from the amount of space 

demoted to the actual analysis of data in methodology textbooks to 
a dp nee m with the forjnat through which instruction is given in 
these text? . One of three different formats,' or modes of present- 
ation, is employed by these texts. The first format emphasizes 
straightforward exposition of the methods, and techniques covered in 
the text . Here , teaching is by talking . Goode and Hatt ' a Methods 
in £ Octal Research exemplifies such a format. The second format 



emphasizes) the exposure of the student to completed research reports 
representing well done research. Here, teaching is by example . 
Lazarsfeld and Rosenberg's The Language of Social Research illustrates 
this format. The third format is a hybrid combination of the two 
previous formats wherein, presentations of abbreviated -and edited 
research reports are followed by an explanation and discussion of 
the research techniques illustrated by these reports. Here, teach- 
ing is by example and talking . Riley's Sociological Research: A 

Cas S* Approach is an example of this third format . 



* • » 

None of the threie formats is too- successful in aiding . the 
student -to develop* skills at discovering relationships in data and 
in understanding how the survey analyst actually goes about his job, 
although the third is the least objectionable/ The expos it ional v / 
presentations fail to get the student involved with actual research 
data! while the presentations using completed research reports suffer' 
from! the fact that it is difficult, if not impossible* .to reconstruct 
the actual steps taken by the researcher during his analysis from his 
formal presentation of the results. In other words, the format of 
reporting research differs-' pons iderably from the format used .to Carry 
it out. 
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\ In summary, while all of. the topics covered in both gen- 

eral and specialized met hp do log/ textbooks should be mastered in 
order to carry out a research enterprise, the most crucial stage of 
' that enterprise — discovering relationships and linking them to - 
gether into comprehensive'’ 'and coherent causal chains — is the one 
thing that these texts most neglect. Not only is relatively little 
space in these texts allocated for developing skills in data analy- 
sis, but, in addition, the formats utilized, in discussing data 
analysis, fail to enlighten the. student about how research is actual- 
ly carried out. Such texts thus emphasize methods of verification 
at the expense' of methods, of discovery, and they focus on the iso- 
lated building blocks of analysis without linking them together into 
a larger framework. Tfte' student , when confronted with real survey 
data, is often uncertain; as to what to look. for first, is undecided 
concerning what ' to look for next once he launches his analysis , and 
is unsure whe? to draw his analysis to a* close. He is placed in the 
position analogbus to a novice, at the game of chess, who has been 
instructed in the legitimate moves each chesspiece may make and 
what are -the overall objectives of the game, but who lacks informa- 
tion about the appropriate strategy and tactics for opening game 
moves, designed to develop his pieces j middle game moves ' to improve 
his position and engage his opponent, and end game moves designed 
to- ensured victory. 



/ Survey analysis is somewhat of a game, and the expert / 

■ stfTvey analyst J' like the master chessplayer, must thoroughly under- ^ 
stand the strategy and tactics appropriate to each stage of analy- 
sis and be able to anticipate several steps in advantfe the likely ' 
consequences of making one analytical move rather than another. . 
Unfortunately, textbooks .dealing with survey analysis, unlike chess 
manuals do not focus on building up the student's understanding of 
such strategy ,and tactics. Our project explored the feasibility of 
simulating actual research conditions to instruct students in re- 
search strategy and tactics, with the hope bhat these new approaches 

could be incorporated into future textbooks. 

* . . - .... * • * 

P * 

' * • ■*» * 

3. Limitations of instructional styles in methodology classes 

. • • * , * f 

Not only . are textbooks, negligent in presenting material 
that would aid students to develop the skills and insights necessary 
to carry out an adequate analysis of .-survey data, but classroom in- 
struction techniques, also, fall in this regard. Upon completion of 
, a methodology course, that included instruction in survey research. 
Students often found themselves still confused and ill at ease when 
confronted, with actual survey data requiring analysis. They appeared A 
unable to discover fruitful relationships which can be developed ^ 
into more elaborate themes with interesting variations Culmindting 
in a coherent, comprehens ive. and cogent~afrplailatorv system! • The 
reasons for this^f ailingin class room instruct ion in survey analy- 
8is_are-seyeral7^ * ' . ^ ! ; 
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First and foremost praong the reasons is the fact that 
little classroom time is given to instruction in the art of analysis 
itself., just as methodology, textbooks devote .little space to %idis- 
cussion of actual 4 s t a , analysis instructors’^ methodology places 
devote little time to the subject . Much more -classroom time is 
usually spent in diSchssing general problems 1 of research, *re search . 
design, questionnaire construction, sampling procedures, interview-' 
ing techniques, coding operations apd : general problems of measure- 
ment, index construction, and scaling': ' Although all of these topics 
are related to the actual analysis of data, in and of themselves, 
they fail to ogive the student sufficient insight into the detailed 
steps o£ data analysis,. More time -should be allocated for instruc- 
tion in the process of data analysis itself. 



A second reason! for the inadequate training of students 
is that what little time is fepent on instruction irt data analysis is 
at times abstract rather than concrete- in nature and expositional 
rather than experiential. What the student sorely needs is practical 
experience in analyzing concrete /survey data under the guidance of a 
trained instructor. Such practical experience is possibly less 
crucial in the teaching of., experiment a! methods where the analysis 
is relatively systematic, codified, routinized, and anticipated in 
the research design stage. In/ survdy ‘research, however, where such 
attributes are regrettably lacking, practical experience is essen- 
tial.. Learning by doing is thus^ particularly relevant in r the train- 
ing of survey analysts. Unfortunately, in most methodology classes 
tojlay, students spend Very /little time in 'doing survey analysis r 
using concrete, real data.’ Moreover, even when such practice is 
undertaken by students, tfiere . is often a lack of 'adequate supervision 
by trained personnel . 
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A -third and /final redson why students in methodology * 
courses are poorly prepared in the art of data analysis is that even 
when they are on occasion given practical experience in analyzing 
concrete survey data, that analysis is. often confined to the simplest 
level. Thus/ students are encouraged to look a‘t the univariate dis- 
tributions of the responses given to the various questions by the > 
respondents and then proceed to detect relationships between some of 
these variables by constructing two variable tables. At times, . 
student s go somewhat further in' "elaborating"' these two * variable 
building blocks of survey' analysis by the introduction ol* three 
variable .tabj.es . Often the -entire analysis* is limited to a handful » 
of twoyariable tables supplemented by one, two, or three variable 
'Cables,./ Even at this elementary level students are often confused 
and not given sufficient time and > supervision 80 develop 'their comp- * 
etence and confidence in data analysis. Moreover, they of ten. gain ’ ’ 
no insight- whatsoever. 'into the more -demanding -task of linking- a 
number- of component two and three variable Cables’ together in order 
to form a comprehensive and coherent analytical structure of sufficient 
scope to tell an interesting sociological "story." The strategies* 
and tactics inquired for such a task are rarely- communicated to -the" 
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Student. Thu£, to resort to our analogy with chess -playing, the *, 
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student la at. best lgb^ted to an understanding and» practicd of t 
"opening" mpves off the analysis, and he is relatively Ignorant of 
the appropriate moves fqr the further, "middle" development. of his 
inquiry and of the "end" moves that will successfully wind up the 
analysis. 



In 'our Near 1 let discussion of textbooks,/ we Indicated the 
different ’formats .used to convCy the material and' the limitations I 
of ea.ch of these formats. Similarly, we will now discuss tha-dif- 
ferent formats used by Instructors in methodology courses and indi- 
cate the limitations of each. In general, there are four different 
formats of classroom instruction in methodology courses that we ' f 
have identified. One relies . solely On lectures and discussion; 7 
another supplements such lectures and discussions with simple 
practice exercises; a' .third calls for the invoivemeht of the students 
in a. major class project which attempts to be a microcosm of a pro- • 
fessional survey operation from beginning to end; and a fourth has > 
students engaging, in the secdndary analysis of previously collected 
survey datav We will now discuss -each of these four formats in more 
detail. • 







The first format we will consider calls for the teaching of 
methodology. courses along the lines traditionally used for most other 
college coursed. Here the -instructor lectures op a single topic each> 
class meeting. Thus, for example, one. or several class meetings are . 
devoted ’exclusively to problems : of research design, or on question- 
naire construction, or on measurement techniques. These- lectures 
are often supplemented by discussion sessions .led by the instructor 
or an assistant during which the ■'students can raise quest ions, con- 
cerning points .covered in the lectures that perplex them. • Hopefully, 
these discussion sessions serve to resolve any confusion in the 
.student 1 s mind as well as supply him with supplement Ay information 
on the topics covered by the lectures. 



* kJS 

• *' / . This approach to the ’classroom beaching of methodology 

suffers in' several respects. To begin' with, since each topic is . 
considered in isolation Qf the Others, it is difficult for the stu- 
dent to appreciate the integrated nature of the research enterprise. v 
Instead, he- often emerges with a segmented and disconnected image of 
research 'activity, failing to see how decisions at one stage have 
repercussions on all * the - subsequent stages-. In addition, this format 
fails to give, the student any practice in analyzing survey data, * 
Students in such courses* are often* at ,a loss when asked to carry out 
even a simple analysis of survey data summarized* in ^tabled. They are 
often bored or frustrated. Such/feelinga might be -partially over- - v ‘ 
come if students are . introduced^tt) the more 'exciting, more creative 
’aspect 8 of research whichjktivolve the’ discovery of interesting rel'a- • 
tiopship8 and linking tjim to form a .causal explanation of some 
interesting phenomena; • „ 







-7- 



10 












... - Format number two is an elaboration of. the first. Here, 
thft instructor supplements • the lectures and discussion sessions with ■ 
a - series of practice exercises .designed to give, the student more 
direct contact with different phases of the research. The students, ' 
‘for example, might construct simple questionnaires , interview one 
another, code open-ended responses .to a previously administered 
questionnaire, draw a mock saittple, see if a set of.; items form p 
scale, -or analyze a few selected tables. Although these activities 
On the part of the students are not integrated or utilized in an 
.actual research ^project, they, nevertheless, giye the studehtsj 
some tangible contact with .specific research operations. , 



This format,' also, exhibits s'everal serious weaknesses. 

-r Like ’the. first 'format, this .also presents the research enterprise 
* as a series of Isolated, operations whose integration is not experi- 
enced by the student J’. While students taught according to this- second 
format do engage ’in practice exercises, these exercises are often too 
brief, artificial, and ’segmented for students to develop much insight, 
oj: skill in research. Furthermore, too little time is spe'nt on prac- 
. tice exercises ^n data analysis; and the students are almost as much 
at 'a loss to carry out even simple analyses of survey data as those 
students taught in courses using format number one.’ Finally, many 
.' of the' practice exercises are felt to be boring or irritating and 
; . are carried oUt in a. reluctant and resentful mood. This further 
detracts from whatever heuristic value the practice exercises may 
have. • ^ 

. / * » ♦ 



The third format entails having students collectively 
carry but a simplified version of a complete research project. 
Students, together with the instructor,' frame a research problem 
that is susceptible to investigation with survey techniques, 
develop a research design that translates the problem into appro- 
priate research* steps, construct an interview schedule, draw .a 
sample to be interviewed, administer* the. schedule to respondents, 
code the responses 'to-the interview, \ punch the coded information 
onto IBM cards, arrange for the^rqpisfer of the data onto computer 
tape, N write programs instructing £he computer as to how to process 
the da^a, analyze the computer print-outs, and write a report based * 
on the analysis. Here, the student does, indeed, acquire practical * 
experience in* carrying out a survey. He gains- experience with indi- 
vidual research operations, and, more important, he- begins to see 
. how they, are Integrated. : 

\ Although this format of instruction is superior to' the two 
preceding formats, it, also, has its limitations. Class projects 
such as these are limited to the largdr colleges and universities 
.with equijJbent and funds . This approach is impractical for usSVby 
. the large. number of siball colleges. Rather than use computers fbr 
data .processing, they* must rely ron hand sorts on McBee cards, ‘or 
.utilize puuched card equipment, j .Furthermore, little time is spent 
in the actual analysis 'of the data by students engaged in such * 
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collective projects. The greatest bulk of their time is spent on • 
constructing a questionnaire,' interviewing, preparing the data for 
analysis, and learning how t& run punched card equipment and/or 
writing computer programs. “As a result, the final reports turned 
In by students from such projects are often of low quality reflect- 
ing the haste and confusion with which they were put together. 
Finally, the brevity of the Interview schedule 4nd the^emqll number 
of respondent? that usually characterise such class projects often 
prevent the students from* emerging with' a penetrating analysis of 
the phenomfena' under investigation. . 

' . * . 

The fourth and final format calls for students ■ to under- 
take an extensive secondary analysis of previously collected survey 
data*. The survey data used is usually of professional calibre — 

• the research design, the interview schedule devised, the drawing of 
the sample and interviewing of the respondents , and the processes' 
by which the data are- prepared for processing. Professional standi 
ards in these matters means that there are fewer, "errors" at each 
stage of the research process. The cumulafiyg effect of iiany errors 
that could seriously undermine the credibility of the analysis is 
avoided. Such a danger is more likely to occur in a 'class research- 
project. Since the data used.for a secondary analysis is often a 
large scale survey, ' the range of information 'solicited from the v 
respondents and the large number of respondents interviewed permit 
much* greater sophistication in the analysis. Sedfendary analysis 
focuses exclusively on the analysis phase of research and encourages 
the student to discover new relationships in the data and to link * 

* them ia order to form an integrated, comprehensive, and smoothly 
flowing analysis. In conclusion, (Jhis format gives the student the 

. greatest exposure to the experienc^of analyzing real survey data . 
Therefore, it is the most useful ofVall the formats in developing 
skills of data analysis. » ° - \ 

Despite the obvious advantages of this fprmat, it, .too, has 
its drawbacks , To begin with,* it fails to emphasize the integration 
of the various steps of £he research enterprise; The student may 
fail to appreciate all the effort and skills that ultimately led to 
the creation of the data that he is using. Although more time is 
spent by the student in actually analyzing data. In courses using 
this format than, in thoSe using any .of >the previously described for- 
mats, d substantial amount of. time is still occupied with non-analy- 
tical tasks which tend to distract one's attention* from analyzing 
tables -- e.g.,* learning machine operation and programming, under- 
standing the codebook, and construction qf analysis decks or tapes. 
Still another drawback Is the fact that even more so than in the 
ease of class projects, secondary analysip- is mainly limited to 
the larger, more affluent, colleges'" and universities where modern 
data processing equipment is available for student use. /it, also,' 
depends on the. possession by the instructor of the raw data from a 
previous study that is in satisfactory condition to be analyzed by 
students. Moreover, personnel are required to assist the students 
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> in punched card machine operation and/or computer' progres^ming. Few 

* small colleges 1 are likely to have the equipment'', material, ,andpers$n- 
• nel v needed, to ca?ry out secondary analysis. Finally , ' the data used * 

in secondary analysis, would have been previously analyzed and, in all 
likelihood, published by .^.professional researcher, and the mos.t~ob- v 
. * . flVious lines of inquiry would have, alreadjy been explored. The student 

would be jforced to focus his attention on questions that are peri- 
pheral to the^ origfpdl research design'. The task becomes mbrq demand- 
ing than would be the ease if students were given an. unanalyzed stifdy. 

' The difficult' task all novices face in coming, to grips* with survey V 

• ' data^-for' the first time is further compounded by the need to restrict 

their inquiry to fhe more hidden, secondary paths. 




i 







4. Heed for neW textbook and classroom formats in methodology 

< - The foregoing sketch ofx'fe limitations and failings" of ' 
both classroom instruction and textbook presentation of methodology * 
highlights the- need. 'for new- approaches. The most cruicial -need 'today 
in Che training of students in methodology is fop new. approaches 
that help the students -develop greater understanding and insight 'into; 
the actual process of data analysis. Such’ new approaches’ are parti- 
cularly needed ini\the training of students' in rfurvey research where 
the research design rarely completely anticipates,. the subsequent 
lines of analysis.- 'This, we believe^ can .best be accomplished by '' 
having the student attempt to- discover relationships in real or simu- 
lated data and *then theoretically, link such discovered relationships 
together into a comprehensive and coherent explanation of the phenom- 
ena under investigation. These new approaches, then, should focus 
^ predominantly on the art of data analysis itself rather than on aux- 
iliary stages of the research process.' Furthermore, they should 
i focus on the logic of discovery rather than on the logic of verifi- 
cation. In addition, these new approaches should be capable of hold-, 
ing the students' interest and stimulating their curiosity. Finally, - 
they/sheuld be able to be employed in small as well as in large col- 
leges and- universities. In other words, these new approaches should 
be- designed to coqgg ag§gg ^ for the- inadequacies existing in the ap- 
proaches cu^xitl|r apffyed in methodology texts and course. 

The development of competence in the analytical skills and 
insights that characterize the professional survey analyst can be 
fostered, according to some, by extensive participation in ongoing 
research. projects. 'According to this view, the only form of training 
that is at all useful, is an apprenticeship type program wherein the' - 
student adopts xhe role of a research assistant. The apprenticeship 
model of "learning by doing" has much to be said for.it; especially, 
when the activity of the student is .closely- supervised by a profes- 
sional researcher. Nevertheless, the major assumption upon which 
our project rested was that alternatives to the apprenticeship model 
could be developed and incorporated, both in methodology texts and 
classroom presentations. These alternatives should approximate, .to 
some extent,, the "learning by doing" experience associated with the 

" - .* •' ' : V . ' . 

: J3 - 

. m y - 






_ . r •- • \ v . . 

4 * ' ^ “ . 

* ‘ • ."■•»* • t% 

apprenticeship model while at the same time having wider applica- 

.. ' tion., They should supplement the approaches already employed in 

methodology courses.- 'These new approaches should aim at reducing, 
the wide chasm that currently exists between the*. training students * 
receive in 'survey analysis and the analytical skills and insights 
actually needed to. carry out a. competent analysis of survey data. 

” » A reduction in this chasm would have two important consequences. It. 

, would encourage a larger number of students to participate in survey 
' research projects' in the future, and it would enhance the value these • 
students would derive from such participation. • , 



The. first, c’onsequence -.- that of encouraging great et part i- 
. , > cipation in survey research -- would come about because students would 
be introduced at an early* point into the most excitingand.ereative 
. aspect of the research enterprise — data analysis* Presumably, this 
jrould '.result in greater enthusiasm -and. excitement about research on 
.** the 'part of students which, in turn, would encourage them to- become 
apprenticed to professional researchers*' carrying out Ongoing^ survey 
research projects. The second consequence -- that of deriving great- 
er value from participating in future Survey research -- would, result 
. from the fact .that students would more likely engage in the analysis 
of data, rather^fhan iti being relegated to routine and less challeng- 
ing- tasks as interviewing, coding, or percentaging. 



~ The 4 recent development of survey data banks,. 'both here and' 

abroad, callsy for ad increasing number of persons equipped/ to engage 
in detailed* secondary analysis of data bank material. The approaches 
developed in this project are designed to help the student become 
competent 'in secondary analysis. * • . / 
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5. Creating, cross -tabulations 

^ 

The basic assumptloi&underiylng ihe approaches developed 
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In. this project Is that students best learn data analysis by actual- 
ly doing It. While discussions of the general principles underlying 
data analysis and exposure to examples of completed research reports 
are of value, they are no substitute for the actual analysis of data 
Itself. Often, however, it is impractical or Inconvenient to plunge 
novices immediately iiito the confusion of the swelling sea-.nf-raw 
Interview data. > Consequently, we have developed comprehensive sets 
of "middle-range" data which fall between the extremes of the raw 
data contained in interview schedules (which are transferred onto IBM 
cards or computer tape) and the /highly selected, polished, statistic- 
al or tabular presentations, found in published research reports. 



, . • .. p 

The "middle-range" -data consists of a large number -of con- 
ventional tables,- each representing the cross tabulation of two or- 
three variables. These tables were Intended to approximate the range 
tables an Investigator might well have ordered in the process of 



of 



analyzing raw survey data. Some of these tables should prove to be 
of little value In the subsequent analysis, while others should be of 
crucial value. This, again j approximates the situation sometimes 
encountered by professional survey analysts whose initial exploratory 
lines of analysis prove, upon investigation, to be useless. The talent- 
ed survey analyst can pull together those few tables that con^jey the 
thrust of his analysis, from the.enormOud' number of tables that poten- 
tially can be ordered. Our "middle-range" data can aid the student 
develop his ability to tell the "trees from the forest." 
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We developed four' different sets of such "middle-range"' 

tabular data. Each set jfocused on a different substantive area. * 
These four areas were (1)- the voting behavior, (2) race relations, 

(3) -student attitudes and behavior, and (4) social mobility. We felt 
that this wa£ preferable to developing tables dealing in only one area, 
as it increased, the likelihood that students would find an area of . 
personal Interest, which would elevate the 'students', motivation and 
Involvement;.' These particular four areas were selected because of our 
familiar it/ with each' and also because we possessed raw data from 
previous 'surveys In each of the four areas. . , 
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The tables for each area were created in three ways. 

(1) Some were derived' by cross -tabulating variable's contained fin / 
real surveys; (2) others were purely Inventions of our imagination^ . 
baded on no real survey data but guided by our.knowledge of what f 
Certain relationships would look like; and (3) > still other tables / 
/.were constructed by^, starting with real data and then "fudging" the;-' 
results slightly ip order to highlight connections for heuristic ’ 
purposes. Each table/, regardless t>£ how It was generated, contained 
the frequency distributions, both In terms of the -numljer of cases 
falling Into each category and the percentage distribution computed 
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columnwise, In addition, each table contained several statistical 
measures of association. 
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6. Formats for presenting tabular data 
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We developed two different foxroA^s witjiin wtiictv> the u m jddle- 
range ” sets of tables were presented to tiie students. In all ca'ses 
our ultimate goal was to have , the studefttsj formulate some research 
problem related Co the/data they, would' receive. They were, then, to 
proceed to discover relationships between variables cross-tabulated 1 
in tables and link' selected tables 'to form an integrated analysis 
and interpretation. In this section ’ r we will describe how we created 
those two formats, ih the following section, we will discuss the 
relative success and /failures cf these two formats. / 






Format number'ofte began by presenting. the student with a 
final- research' report of profess ionaical ibre .'/Some' such reports ,• 
were created especially for this project, while others were taken • 
frdm the professional literature'. • Reports gathered from, the latter 
source were^jsiliher /presented as originally published, pr were altered 
.somewhat fpr heuristic purposes.. All of these reports, regardless’ 
of their source , contained numerous tables interspersed within the 
text. ^The report /was immediately followed by some commentary con- 
cerning" alternative possible interpretations of the tables. An 
attempt was made /bo indicate why some such interpretations were 
preferable to others* This, in turn, was followea by a mythical 
discussion betifesSn myself and the author of the report, And the prer 
sumed path taken iff the analysis was recapitulated step by step. The 
discussion helpb the student appreciate the. many false starts, dead *-... 
ends, changes in direction, and un- anticipated turn of events tha't 
take place inbeal-life research. The student begins to understand 
that the. ; ordat4y and logical sequence of findings presented in the 
'■final report is a poor reflection of the mental processes and data - 
manipulation that take place during the ongoing research enterprise. 

We, then, elaborated* on different ways in whi.ch the* origin- 
al data could have been integrated ■, and alternate interpretations of 
the phenomena -under Ihve.stlgat ion resulted. Some selected tables 
were juxtaposed for* the purpose of il lust rating .-some of these alter- 
native, line/s -of inquiry. Thus, the student was presented with con- 
crete examples of alternative versions of the final report. *■ 

, '■ ' *> ' "• * * 

Next', the student iras supplied with a set of two and three 

variable tables., which were derived from ■the same data that were the 
basis f of: the tables appearing in the original report. He was in- 
structed to do two things with these tables. First, he was to select 
the ones which best -supported the alternative lines:of inquiry that 
‘we suggested were possible. Here*, the student* is given a "story" and 
has to fit some data* to it in the best possible fashiop. Second, 
given -the. set of tables, he was instructed to develop sorae alternative 
.lines of inquiry of his own; Here, the student is given some data and 
has to uncover a "Story" contained therein. ' The student, together 
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with thp instructor, then, compares and contrasts the various ^lterna 
tive M stories" developed, and criteria are presented , for deciding 
which/of these alternatives' are superior to others. '■< 

The students were then supplied with an extensive set of 
tables, either real or fictitious, which dealt with a topic related] 
to the one discussed in the original report. There were, then, ’ 
several exercises' that the students had to perform with this, data.' 

To begin with, 'limited hypotheses and rudimentary "scripts" were 
supplied along yrith the set of tabies, and the students were tbld 
to discover which tables, if any, supported these hypotheses and 
"scripts." The^ students were also requested to develop their own 
hypotheses and ^script" outlines after looking at a few tables. In 
both cases, the students were then instructed to look at additiona 
tables that jtould either tend to confirm or deny the initial hypo 
theses and ^scripts." At first, thefywere told to hold on to the&e 
hypotheses tenaciously. They were to explain a.way as- best as they, 
could any empirical d&ta that contradicted these hypotheses, a4 well 
as to search energetically for additional empirical confirmation. 
Next, they were told to discard gradually stime of 'these Jiypotheses 
and formulate new ones if the empirical data appearing in the tables ' 
failed to substantiate them.' Finally, they were told to discard 
these hypotheses the very first time some empirical data, failed to 
support them and formulate alternative hypotheses. These exercises 
were intended to convey to the student the capricious, nature of the 
link between theory and data. . 

The Vast step was to have the Student use the. set of 
tabies given him for the purpose of developing an extensive "story." 
The student was\ to link* a cumber of empirically substantiated hypo- 
theses into a* coherent interpretation of the phenomena under investi- 
gation. He was; also, told to list any additional tables which he 
would like to have included in order to further substantiate, his 




analysis. 
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' The approach 'oaken in this first format •was - to guide the 

student in gradual stages from the finished report back to raw data 
in tabular form. We started with a well structured and organized 
interpretation and by stages introduced greater entropy until the 
student was brought face to face with disconnected tables of data. 
The second format we employed' proceeded in the opposite fashion. . 

We initially exposed ^he student to a large set of raw data in - 
tabular . form and then in gradual stages guided him to the finished 
report created by a professional researcher. The student begins 
with, an extensive set of tables, scans throu’gh them, and starts to 
develop a "story" outline suggested by the data. He, then, is 
requested to develop more limited hypotheses and "scripts" from this 
data.' He iSj then, given several feasible hypotheses and "scripts" 
which could derived from the data which he then compares with 
those hd previously created. The student is then required to- put 
together a sequence of tables that will best support particular 
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hypotheses and "scripts" that are supplied him. Next, the student is 
confronted with *a reported reenactment of the mental processes a pro- 
fessional researcher might go through in trying to tease q "story" out 
of the .set' of tables supplied. Finally, the actual finished report is 
re ad by the student. In brief , format two- is really format one stood 
/on its he'ad . ** . 

Each format contained some novel approaches in the teaching 
of data analysis. We were concerned both with how these different, 
aspects went over as well as with the question of which of the two 
formats were more effective teaching devices. In the following 
section on if indings , we will attempt to answer these question. Be- 
fore we do this, however, let us turn to several other heuristic de- 
vices we developed. * . 

7 . Games I , 

( * * 

f df v 

In addition to the formats discussed above, we also devel- 
oped several heuristic "games" designed to introduce students to the 
art of data analysis. We purposely inserted- a competitive aspect in 
most of these games in order to heighten motivation and interest. 

The students played at the 'various games before being exposed to the 
two more extensive training formats* Prior to playing these games, 
ail students received rudimentary instruction in survey 'analysis. 

They were instructed in the following concepts and techniques: Distri- 

butions, cross -tabulations, 'correlations, "elaboration," and multi- 
variate analysis. Instruction at this stagfe was more condensed and 
focused than is Visually the case, and we purposely ignored a, number, 
of issues normally discussed. Our intent was not to have thie students 
thoroughly master these, various aspects of data analysis, but merely 
to: gain a superficial' familiarity with the terns and lo^ic of ' survey 
analysis. A more thorough understanding of these matters, we hoped, 
would come aboht after exposure to the material we "had developed. 

Following- this brief, instruction period, the class was 
broken up into small groups, and each group was divided 'into two 
teams* These teams then played several games we had developed express- 
ly for heightening their ^nsight into the art of survey analysis. A 
script containing a "story" based on data and empirical interpretations 
had been given to each team for study before class convened. One team 
was instructed to try its best, by use of the empirical interpretation 
tables (which were on cards), to substantiate the interpretive story 
line developed in the script; the other team was instructed to try its 
best to refute i;hat story line. * 

.-During the class period, both teams were , to discuss the mat- 
ter, and each team would attempt to win over the other*. Each group of 
two teams made 'its presentations to the entire class. One team argued 
for the script and the data; the other attempted to demonstrate that 
the data %(a s ill suited for supporting the story line emerging from 
.the script.' After both teams had engaged ifi thorough debate', the 

■ - 15 - ' / • '■ ■ ■ 



18 



t 



instructor .led the class in a discussion on the merits of the argu- 
ments presented by each side. ' 

The winning team received a certain number of positive 
points; while the losers were penalized with negative points. If, 
prior to the class presentation, one team had wisely altered its posi- 
tion in light of/ the other team's arguments, it was not penalized. 

On the other hand, if the class as a whole felt that its original posi- 
tion had merit and was defendable, the team that had shifted from that - 
position was penalized to a greater extent than it would have had it 
remained steadfast -in its convictions. After the class had reached 
a judgment concerning the cases presented by each team from the first 
group, two teams from a second group made their presentation. ’ This 
routine continued until all of the groups were heard. This process 
often took several consecutive class meetings. * The winners of one 
group played the winners of another group; whfle the losers of each 
of these two groups played against ^each other.' This continued until 
most teams had accumulated a certain set 'number of positive points in 
the scoring. Once h team had reached a predetermined scoring level, 
indicating its competence in elementary analysis and presentations, 
its members were free to move on to the more elaborate variations of 

the game. There were several such variations. / 

• * * » » 

. Other games we developed did not require team competition. 

One such variation required each team to enunciate alternative "scripts" 
that might have been written to fit the tables. They were then judged 
according to how believable their script was compared to the original. 

A second variation called for each team to receive at regular interv- 
als, additional tables,- and they werefl to determine whether -or not these 
additional tables altered the fit between the "script" and the data, 
and, if so, in what ways.. A third variation gave each team several 
scripts and a large number of tables. Each member of a team' was* 
given a script, which, he read aloud; while the other, members searched 
through the tables in an effort to find those that would substantiate 
the story line contained in the scripts. 

^ «’ h Another was our "domino" game. In this game, the class was 
again divided into small groups. Each of. these groups was given a set 
of related cards of one particular sociological phenomenon and which, ' 
also, represented the relationship between different pairs. of variables'. 
Each student in .the group then received an equal number of cards , and 
the game began. 

• m 

The game commenced by placing a special octagonally shaped 
card that contained* four pairs of two-variable relationships in the 
center of the table. The first student attempted to "match 1 " one of 
the cards he held in his hand to one of the eight si'des of the center 
card. Then, each student in turn would attempt to add one of his 
cards to^the emerging structure created by the linking of sequence 
cards. One end of each card represented one of the two variables in 

the relationship; while the other end represented ’the second variable. 

• ■ * 
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All of the cards were limited to* representing two variable relation- 
ships." The center of the card indicated the nature of the relation- 
ship. This .relationship was indicated in. two different ways:, (l)'by 
correlational coefficients and (2) by cross-tabulations. One could 
add to an already discarded card only by joiniiig a variable at one' of 
its ends with an identical variable in card one still held. If during 
his turn, a player had no card and he felt he could add to the emerg- 
ing structure, he would then pick up one or more additional cards from 
a common pool of cards until he could discard one. Thus, the play ^ 
proceeded along lines similar to those used by -children in playing 
dominoes. Unlike dominoes, however, it was possible to break apart 
two previously connected cards for the purpose of inserting' a new 
card between them. - The positioning of the new card in relation to 
these" cards already discarded indicated diffetept implications in the 
flow of causality. * The "winner" of the game was the first person Who 
discarded all the cards he had held. ■ „ . • 



, The students were to verbalize about (1 the "story," they were 

spelling out in the domino game, For example, one card may stiow a 
relationship between religious affiliation and- voting behavior. A 
second card, attached to the first, indicates that, there is a relation- 
ship between class and partisanship, .while a third attached card shows 
that there, also, is, a relationship between class and vdting behavior. 
The .students might interpret this to mean that class factors intervene 
in interpreting the original relationship between religion and parti- 
sanship. At this point, however, it would be pointed out to them that 
they would need a three-variable table to substantiate this interpret- 
ation. r 



\ 



. - A separate pool of thsee -variable tables was made available 
to each group^ from Which the students could then search for the appro- 
priate table, that would support their interpretation. . . „ 



-Our last game involved the creation of three different s^ts 
of cards. One set of cards contained two and three variable tables; 
a second set was composed o'f analyses of these tables; and a third set 
was made up of more general interpretations of these analyses. The . 
class w^s divided into several small groups of three . students. In each 
group, one student received the table camps; a second, the analysis 
cards; and a third, the interpretation cards. i The student with the 
table cards would' begin by discarding one of Ais cards. Next, the 
student with the analysis cards would sparch^for the one which best 
analyzes the table apd proceeds to discard it. Finally , ■ the third 
student would attempt- to discard one of his cardp which adequately 
interprets the analyzed table. After. all the iards were played, the 
students would exchange sets of cards And begin again. 
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Ill* Findings 

• ’ ■ • 

How effective were the various devices we developed in 
heightening the students' comprehension and skill in the art of • 
data analysis? In order to answer this question, we initially intend- 
ed to enlist the aid of colleagues at different universities who were 
teaching courses in survey methods courses in either sociology or 
political science departments. They were to use the materials in a 
selective way whenever possible, so that a clearer picture of their 
effectiveness might emerge-. For example, students in a class were 
.co be divided into a number of different sub-groups on a random 
basis, and different materials were to be used by the students in 
each sub-group. .All the students in a -class were to be given identi- 
cal examinations designed to test their ability to analyze actual sets 
of tables. In addition, the -students were to engage in a brief analy- 
sis of conventional raw survey data. The relative abilities of the 
different students in data. analysis, as measured by both their grades 
on the standardized tests and their performances in analyzings actual 
survey data, were to be correlated with the different materials to ‘ 
which they were exposed. Thus 1 , our evaluation of the relative effec- 
tiveness of the materials which we had developed depended to a large 
degree upon the systematic feedback of' information from colleagues 
utilizings these 'materials in methodology classes-. 

"• v •, t 

We should. point out that even with our own classes, limits* 
tions of time and personn&l made it impossible to rigorously follow 
the "testing" format suggested^ above. Hence, the. findings reported 
■±n this section largely rest on our sub jective impressions concerning 
the relative effectiveness of the materials presented. Nevertheless, 
considering the "exploratory" nature of tjiis project in. exposing 
students to a variety of novel approaches in the teaching of data 
analysis, our impressionistic findings may not be a liability. 

d - ‘ . . 

Flexibility, rather than rigidity, is important at the :; 

exploratory stages of a project. This flexibility facilitates jthe 
discarding of approaches which seemed promising earlier but fail, to 
live up ta expectations . Flexibility also entourages tinkering with 
the!; original material in light of experiences -with it. in actual class- 
room use, so as to improve its value. . It, also, promotes the spon- 
taneous development of new, unanticipated materials suggested by our 
classroom experiences. A. preconceived experimental testing design 
mitigates against such flexibility in favor of Scientific rigor. 

Such rigor is more appropriate at a later stage when we -want to choose 
from, among several alternative sets of materials , which seem from 
impressionistic exploratory probes to have some value. \ 

» * 

. Let us now turn to the impressions %fe formed concerning the 
effectiveness of these materials in training students in. the art of 
data analysis; , We will begin by focusing on our experiences with, the 



(several data analysis games we developed and discuss the* effective- 
•ness of the two training formats. • . * 0 

8. Examples and analyses of games .* ,, 

In all our methodology classes, the studdnts were briefed 
on the rudiments of survey, analysis before proceeding to our special- 
ly designed games . v How well did this Instruction gdtover? The first 
trial class reeled from having too much information thrown at them. 

The students felt that 'they had not coped adequately frith one topic 
before they 'were in the midst of discussing another. .With our second 
class, we reduced, the amount of material introduced in the beginning. 
The topics that had been eliminated were introduced late,r at inter- 
vals. Furthermore, rather than begin by discussing methodology points 
in , the abstract, we began by posing concrete' problems in different 
substantive areas of sociology. We then proceeded to show the stu- 
dents how different types of data handled in different ways can throw 
light upon these substantive problems. This problem-oriented approach 
proved much toore effective than the abstract approach in introducing 
students to the essentials of survey analysis. At this— stage, the 
students liad not mastered. the art 9 f Survey analysis. Nevertheless, 
they had acquired an appropriate vocabularly, .understood the rudi- 
mentary logic of survey analysis, and began to appreciate the value 
€ <*f methodological points in clarifying real world problems, both 
‘practical and theoretical. \ 

I • • . * 

Approximately six meetings of our methbdology classes were 
taken up with a discussion of these basics in survey analysis. The 
’students were then introduced to the games . How well did these games 
go over? What alterations were made in light of our experiencws? 

And how effective were these games, in realizing our* purpose? The ■ \ 
games j by and large, were well received. Students appeared to enjoy 
the competitive and novel aspects of the games. With our novice stu- 
dents, these games were, more successful in holding interest than the 
conventional methods of teaching data analysis. Let us now turn to 
a discussion of what alterations we made In these games. 

The first game involved dividing-the class into skveral 
groups of two teams 'each . Each group received a script , which elab- 1 
orated a story line, and a set of tables, which formed the empirical 
basis of the script. The teams played against each other. One team 
rried to. demonstrate that the 'data presented ip „ the tables supported 
the story line in the sqript, and the- other team attempted<to show that 
the^story line was unsubstantiated by the data. What impressions did 
we- form on the basis of our experiences with this game? 

* ’ ’ •/ ' \ 

'First, this game consumed much more time than we ftad intend- 
ed. In subsequent trials, we reduced the amount of time needed by 
introducing the following alteration^ : ( (1) The groups were made larg- 
er, so that there were fewer steams making thfeir presentations in ciass; 



(2) the discpssion between* the teams in each grbup now took place ' 
outside of the regular classroom, either in laboratory sessions or 
at someone's residence; and (3) we divided the class -into two parts, 
and teams made their presentations simultaneously to one or the other' 
half of the class, rather .than ‘have every team present its arguments 
before the entire class. As a result, the total class time consumed 
by this game was' kept within reasonable bounds. 

A second impression of our experiences with this game in the 
early trial stages was that the students-'^ class presentations 1 ' were 
weak and confusing. We felt, this -was due*' partially, to insufficient 
time allowed for discussing and preparing the presentation and to a ' 
lack of supervision, in developing such’ a presentation. * In order. to 
increase the time each team spent outside of the'clnss discussing the 
script and the tables, we had more copies reproduced, this enabled 
every team to have its ppn copy Of both the script, and the tables, 
and meetings were arranged more readily and frequently than hdd been , 
possible with the larger groups.' In order to give the students super- 
vision in preparing their presentations, each team met at least twice! 
with a teaching assistant. The assistant participated- more as a 1 
group facilitator and Information source rather than as an autho^ita- - 
tive leader. These changes brought about higher calibre presentations 
in -succeeding classes. • • , 

• - ■■ ; 

Third, ttfe involved point scoring system wh^ch we had devel- 
oped proved confusing And irritating to the students and time consum- 
ing for us. After a while, we, had the teams play against pne another 
without keeping^ a record of' their* scores. Nevertheless, "winners" of 
each contest played against one another^ in.a round-robin fashion un- 
til one team had triumphed over all previous "victors."’ The -members 
of that team then went on to piay some Of -the other games among them- 
selves, while the other teams continued to contest each other until 
each was victorious over the remaining teams. The competitive spirit 
was retained,, but the. burdensome point system was' discarded. This 
method allowed the "losing" teams to have, more practice in playing the 
game while the "winning" teams enjoyed the opportunity to play the 
more individualized games. 



-Finally, we noted that Some of the teams were dissatisfied 
with the assignment to either defend or refute the story line ’con- 
tained in their script. During later trials, we gavc^ each team the 
option of deciding among themselves which position they would like to 
defend after they had had An opportunity to inspect both the script 
and the tables. Although this procedure resulted in greater interest 
%$nd motivation, it suffered from two drawbacks. In the first place, 
an uneqqal number of teams chose a particular position; consequently 
the ^'pairing of teams was difficult to arrange. Secondly, too -much 
time was consumed by teams deciding which- of tlft two positions they 
wished to defend. 



In the future, a compromise arrangement would be best. Each team 
would be assigned- a position, then allowed a brief time in which to 
decide whether or not they would chose to 'defend that position. If 
not, they could alter their position if they would find another team 
to trade position with them. In thi's^ way , ' it shduld be .possible to 
maximize interest and still have an equal number^ 6£‘ opposing teams. 

After a team had defeated the remaining teams in the round- 
robin contest, its- members began to play the other individualized ' 
games among themselves. The first -of these called for the students 
to create new scripts from the tables which their team had received * 
earlier. Each studbnt ya% to-do this Independent of the others, and 
after each had composed such scripts , he would discuss the strengths . 
add weaknesses of the scripts with the other' members of the group. 
Despite seyeral attempted variations, the Students experienced cbn- * 
siderable difficulty in creating appropriate scripts. Thus, at this • 
stage in their training,, students were much more hesitant about ppin-' 
ning stories from datd than they Were in relating data to stories 
which had already been constructed. 

• . ' • ' v* ^ i 

.All .teams wer^. later given additional tables, and each 
member of ‘the team had to indicate in which ways the additional data 
might alter the previous .fit between tables and script. [Some of these 
additional tables were 'purposely designed to refute the story line 
developed sin the- script; others were designed to buttress tha%story \ 
line. ' This exercise sharpened the analytical ‘.eye of « the student, afd 
made him appreciate the way in which additional information can alter 
one's -empirically grounded, notions. Although 'the students' had soine 
-difficulty yith this^ exercise at. first, they later did very Well and' 
appeared to enjoy it thoroughly. -T ■’•••• .'!! 

- * « •, . • * •' - . - / 

•. The^ext game involved having each member' of a team read 

.'out a. script, and the other members searched through a pool of tables - 
jto 'substantiate the story line developed in the script. This game 
Worked . reasonably well. We added the requirement . that the students 
‘also search -for tables that would refute the story line contained in 
x the Script. After 'some trials, we arrived at the conclusion that the 
best way .to utiliee this game was to divide the members of each team 
'into three groups-. One team was comprised of those who would attempt 
to refute-, the story line; a second team, of- those, who would; try to., 
substantiate it; and a third composed of that individual who was read- 
ing the script.- In-order -to '■facilitate the processing of the tables, 
We* gave -each team two sets of the : tables,- one for each group arguing 
for or against ther- -story line - developed in the script. 

• , 1 . ■ , ' •’ f. ■ , . 

- The* students .were later divided into small groups , and they 
played bur -domino game using as dominoes-, cards embodying two^ variable 
relationships pertaining to a single sociological phenomenon. This 
game caused ebme.perblems among .the students at first. We made a . -- 

number-of alterations^ To' begin with, rather than have a whole group 
play together , we restricted the play to two students at, a time. This 
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caused l^ess' confusion among the students, reduced the waiting time 
between successive moves by the same student, and increased, the like- 
lihood that tabl'es that one student needed were not already appro- 
priated by others. Second, we had " all the. cards in the pool turned 
face up, so that students could pick only those cards that they wanted 
This reduced the number of cards a student Reid in his hand at any one 
time. T(iis, in turn, reduced the student '& confusion. Third, the' * 
."hands dealt to each player were pre-detetmined by*us, rather than « 
left to chance. This* made it Easier for each player to develop a 
story line with the cards in hand. Finally,' to each pair of teams 
playing ^-the games, we assigned one. of the better players. His func- 
tion was to pass judgment on disputes as well as to be in “charge of 
the set of three variable tables. We are stilV not complete ^Satis- 
fied with' the current state of this game, but its usefulness Shears 
much improved from what it was at first. 



■ t ' The, last game involved the use of cards symbolizing the 
three different levels in the processing data — tables, analysis of 
tables, interpretation of the analysis, fcne student would discard a 
table-card; the next student would thrown out an analysis-card that 
corresponded to that table-card; and the third, student would discard 
a card which best interpreted the analysis' - ^ the tab If. The students 
performed quite well at this game'. Nevertheless, they appeared bored 
and as if they , felt they had mastered these skills earlier. We be-, 
lieve that this game should be' the first, orather than the last, pne 
encountered by the students. *. ' 

. : 1 * \ ' 

How effective were all of these games after above, alter- 
ations and modifications had been- Inade? * In our judgment , they were 
very effective. After completing the entire beries of games, most 
students appeared to understand the ways in which survey data can be 
used to confirm or disprove hunches, hypotheses, and theories,.. They 
Also appeared knowledgeable . in the ways in which a sequence w tables 
can form the empirical foundations for an involved explanation of a , 
sociological phenomenon. Furthermore, they gave every indication , of 
being at ease with tables and data, rather than intimidated and awed 
-- a characteristic reaction among novice methodology -students. 

When several selected students at this stage "ordered" table* from 
actual data that was computer processed, their understanding, ^compe- 
tence, and confidence were considerably above those of students ex- 
posed only to' conventional survey analysis training for the same 
period of time. . However, the students still had difficulty starting 
with a set of tables and developing a story line from rtetaw ' This . 
skill, we hoped, • would become developed when they were/e&posed to 
our training .formats. 

U . f ■ * 

9. Examples aiid analyses of formats 

fs 

The students were given ample opportunity to play the 
variable games. Afterwards,. they were exposed to the two formats we 
had developed earlier? The purpose of .these formats was to sharpen 

•# * " . 
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the analytical Skills that students developed in the process of “play- 
ing the games. These formats showed how middle-range survey data, 
i.e., tables, were integrated in order to form a finished research 
report. The roles of assumptions, analytical skills, and interpre- 
tive flare were emphasized. The students were shown how tables are 
woven in gradual stages. One format began with the; report and ended, 
with the data; while the other format began with the data and 'culmi- 
nated in the report.. After a student read through the format and 
engaged- in some of the exercises, he pursued, his own analysis, of the* 
large set of supplied tables. Let us now discuss our impressions 
based on our experiences in developing ’and using these training 
.formats. • ; . > 



The tables, as well as the reports, were created in one of 
.three ways -- out of our imagination, out of,/ actual data and reports, 
and combination of the two. In our ‘judgment, the third alterna- 
tive (combination of imagination and actual data and reports) . worked 
out best in classroom use. When we limited ours elyas to using actual 
data and reports, it was difficult to find exactly what we wanted in 
order to make methodological points.. Relationships between variables, 
were' rarely striking enough -to catch the eye of the novice analyst, 
and it took considerable amount of time and effort to grind out the 



appropriate tables .that would! serve our purposes from the raw survey 
data. Nevertheless , the students appeared enthusiastic about the 
idea of using real data. They felt they were learning about real 
>jpaople i lilto'ey were less enthusiastic when fictitious data and' reports- 
- were usetf? Creating these fictitious tables and reports proved time 
"consuming, and, at t^mes,. we created tables which were not consistent 
; despite efforts tp avoid this pitfall. Heuristically, however, these 
tables atyi* reports were ideal for making methodology points clear to 
the students. The third alternative — that of starting with actual 
tables and reports and the altering them slightly --combined the 
advantages of the first two alternatives. Student enthusiasm re- 
mained high, the tinje spent in preparing the materials was less than 
in the. other two procedures, and for the most part, the tables were „ 
consistent with one another, and the methodology point s> we wanted to. 
make were clear. ' 



- ■ ' • % 

We propose that in the future our middle range tabular data 
- be prepared in only one, rather than in four, substantive areas. The 
reasons are several. To begin with, it took considerably more time 
to develop these formats than we had anticipated. When 'faced, with 
the task of developing four such foinnats, our time and effort were 
divided, and not enough attention was paid to developing a format. 

If we limit ourselves to creating only a single format, less total 
-time should be absorbed in this task, and more attention and care 
could be devoted to a thorough construction of this single format. 
Also, interact ion among the students, instructors, and. teaching 
assistants should be enhanced by having the entire class work on only 
one 'format. 'Teaching assistants could be used more effectively in 
- dealing with questions when the* entire class . is working on a single 
'format. * 
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A number' of the students complained that, 'among the set of ^ 
tables -we supplied there were too mapy useless ones . Few useless 
tables should be included among the set of tables the student is to 
analyze at first. ’This will minimize his confusion and of being over- 
whelmed by the 7 data. " Once the. student gains some confidence in his 
ability to analyze the dat'a, he can be given another set of tables 
which does contain many useless tables-. In light of our experiences 
with these formats, we, also, feel that at the early stages; students, 
should v^ork with only a few tables. The number of tables can be' 
gradually efrlarged as his confidence and skill grow. 

* ./ • f 

The first time we 'utilized these formats, we asked the 
. * Students to pay attention both to the percentage point differences 

'and the correlational coefficients' contained in each ft able. This * 

. proved confusing to the students and hindered rather than aided the 
development of their analytical skills. Subsequently, we had the 
students ^ focus exclusively upon the percentage point differences con- 
tained. in each table ignoring all other statistical measures. Only*' 

, ,; r after the students were able to demonstrate that they could competent- 
"*■{* ly analyze the tables and spin ,off empirically grounded interpret a- . 

... tions were they tpld- to pay attention to the correlational coefficients. 

^ They were then told to retrace .their analytical steps, relying solely 
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on these statistical ’measures J This enabled them to comprehend the 
relative advantages ^a-nd disadvantages of correlational coefficients 
compared to percentage point differences. Finally, they were to anal- 
yze the tables by looking at the correlational coefficients alone. 
They/were then to repeat their analysis having recourse to the per- 
centage point differences shown for each table. These exercised were 
vexy useful in helping the. students develop an appreciation of the 
strengths and weaknesses of different analytical, procedures . . These 
exercises, however, consumed a considerable amount of time. 

*The students looked with favor .upon the discussion style of 
presentation between the supposed author of the report and ourselves 
' during that stage when we attempted to recapitulate the actual creat- 
ive process that culminated in the final report. They were somewhat 
overwhelmed and confu^pd by the extent and variety of. the analytical 
"gig- gags" that -took place in the research process. This was, also, 
the case when we confronted them with numerous alternative lines of 
.investigation that could have been followed by the author if he had' * 

• chosen. It is our feeling that this confusion can be reduced sub- : 
stantially in the .following ways: (1) Spelling out fewer alternative 

lines of investigation or analytical turns and twists by the author 
pf the report and (2) interspersing the discussion between the pre- 
sumed author of the 'reports "and ourselves with pertinent tables that 
"would empirically illustrate and highlight the alternative turns 
' that, were taken. ' 1 

’There appeared to be no clear cut preference on the part of 
t:he students for on.e type of format over the other. Some students who 
.were given formats> that led them from data to the fiqal report ex- 
pressed regret that they were .not assigned the other format. On the' 
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other Hand'll a like nujnber^who. received formats that started with the 
final report and end^d with data Similarly thought that they would 
prefer the alternative format. Most students appeared content with 
the type of format/ they received. From our perspective, it did not 
seem that one typ.4’ of ' format was more effective than another in dev- 
eloping the students' analytical skills. 

' \ 

‘ t How effective were these formats in realizing their dual 
objectives -r of (1) increasing the student's awareness* and under- 
standing of the intellectual turns and twists that link the final 
report of an actual research enterprise with the raw data initially 
collected and (2) improving the student's skill in interpreting 
complex, causal models emerging outt of the analysis of tables? In 
our view, the format^ were very .successful in achieving the first 
Objective and somewhat less successful in realizing the second. .The . 
students did, indeed, grasp the fact that the order of the findings' 1 
presented in a final report bof§ but a faint resemblance to the intel- 
lectual gyrations 4nd data manipulations that 'occur red in the research 
process. They began to appreciate the fact that research involves a* 
great deai more t/han the mere following of set routine'procedures by 
which findings ate ground. They, also, started to understand that 
the art of research- is* much more creative, challenging, and absorbing, 
as well as frustrating, than they had originally imagined. Finally, 
the' students began to see the importance of the researcher's implicit 
assumptions ahd explicit decisions on« the outcome of the final report. 
The roie of such assumptions and decisions were not clearly appreciat- 
ed by the stfcdents 'during their initial reading of the final research 
report. / / , ■ • 



' * .The formats, also, improved the student' s own ability to 
link data/ with interpretations. Many students still encountered 
difficulties, however, when they attempted to spin off ‘elaborate and 
coherent; stories on the basis of their data analysis. However, their 
performance in this regard was clearly superior to that of students 
who received only traditional instruction.' The formats were effect- 
ive in improving the student's ability to move from raw data to 
polished report, but not quite as effective as we had hoped. 

' . ' / • ' • 7. ' . ■ ; ... . : •' 

a; Performance may improve i-f a)f ter Wbrking through the for- 

mate , each student is given a carefully selected set containing only 
a few tables. Then, after he demonstrates his- competence in devel- 
oping interpretation? of different combinations of these tables, he 
ttould be given a few additional tables to add to his set. In- gradual 
stages, he should be encouraged to develop more comprehensive* and 
involved' interpretations and should not be permitted to' advance to 
• the next stage until he has demonstrated his ability to carry out a 
/ satisfactory analysis with the tabled at his disposal* This proce- 
dure' should build the student's confidence and encourage him into " 
handling. larger and larger files; of data. ^ 
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\l0. Questionnaire construction * • ■ ' 

j •• • • 

.. ' The gairies and formats discussed above were designed to a£d 
* fhe student in developing analytical understanding, insights , and , 
abilities. This was, after all, the major objective of our project. 
^Initially ,; we had not plann'ed to developany materials designed to 
improve other phases of the research — e.g., interviewing, question- 
naire construction,' or coding. It became clear to us that it would '*■ 
be important to* sensitize the students to the impact the construction 
of questionnaires have on the analysis of the .collected data. We ' 

' developed several exercises for this purpose. 

' ■ ‘ " * f * 

. . • v ^ r ' , * 

First, students were given a s6t pf tables and asked to 
substantiate given story lines' where either (1) important pieces of 
information were absent becaus'e they failed to have been asked or 
(2) it’wds impossible to carry, out crucial tests concerning alterna- 
tive explanations because 'vital questions had not' been asked .during 
' the interview. In these cases, a the process of analysis serves to 
point out’ additional, new questions that should have been asked .of the 
respondents . The students are told to* look out for -such negligence 
in their analysis and to write up such questions and insert them in 
'the printed questionnaire that is supplied, them. Students were, also, 

. encouraged 'to make up additional questions that should have bepn’ 
asked in the interview that would .have improved their analysis^ In 
another exercj.se-, students were told -about a problem area And asked 
to design a bri&f questionnaire. They were a^ked to write .out a 
request for the tables they ’would want in order to launch an anal- 1 
ysis based on these questions'. The teaching assistants then made up 
, some mock tables that gave empirical flesh to*. their skeletal request 
form. The students were to analyze the tables and then indicate what 
additional questions should have been inserted into their Question-? '• 
naire. This exercise shows the difficulty encountered in -anticipat- 
ing the questions to ask at the questionnaire design stage. The stu- 
\ dents were alpo given -the results of slightly differently worded, but 
similar questions.Jfrom different surveys taken at different dates. 

The diff iculties encountered there in’ arriving at reliable trend 
analysis illustrated the. results obtained with slight alterations ’• '* 
in wording.' _ 

■ Is:',. 4 -' - - ■_ ’I > • y V • . .. ■ . • • _ ** * .... * 

^ These ^exercises proved^ effective in se?nsit;izing the student 

in the relationship between data analysis and questionnaire construc- 
tion. Awarenesq, of the problems in.quhstionnaire construction was 
heightened . The x students . no longer saw these problems as isolated. 
They began to appreciate the advantages of anticipating some of the 
analysis, during the questionnaire design stage. - 
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A hallmark of modern \society Is the support .^'and faith it. 
dislplays towards research .^IHiring the pas k three decades the United 
States witnessed, a dramat id ^growth of research activities in the 
:|Lal and behavioral sciences aa well as/in suc^related fields as 
:ation, welfare, and administration. In response to this un- 
:edented| demand for research im these fields ,the curriculums 
geared to social science majors atAboth undergraduate and graduate 
levels have not i^aBle expanded their methodology 'offerings, and 
there has beeii'a corresponding increase in the number of method- 
ology textbooks designed for social science students . The increased 
exposure of social science students tOvinethodology courses coupled 
with! the rapid multiplication of methodology textbooks might lead 
one tto conclude that the social scienceXstudents are being adequate- 
ly trailed in the art of data analysis. Xunf ortunat e lyL this pre- 
sumption is unwarranted. This is particularly true in the case of 



survey analysis, which today constitutes ol 
social research. ’ , " 



of the major forms of 



Unlike experimental data,- the precise paththrqugh which *! 
the slirvey analyst; travels is not adequately foreseen in ttye research 
design stage. ^iThe^urvey analyst finds himself inundated- with data 
from which tens of thousands of potentially useful tables can be 
created.. Telling the "trbes from the forest" is\an essential talent 
in such a situation. j The survey analyst seeks aXpath through, the 
elaborate and confusing maze of data tha^will lead to a coherent and 
revealing description or explanation ofj tjfe phenomenon under investi- 
gation] The search for such a path calls for the. development of in- 
sight into the. methods of discovery rather than to .methods of verifi- 
cation.! Unfortunately ,' classroom instruction and textbook presenta- 1 
tion in! American universities today often focus exclusively on methods 
of vgriEication. They fail, by and large, to develop the student's 
insight 1 into: how to go about discovering relationships and 'linking 
'-them to \ form complex and coherent explanatory structures^ The develop 
ment . of Wuch insights, however, is an essential ingredient in the 
- trailing!. o f * nov ices. 'in thlrart of survey analysis. ' ^Netf materials and 
approaches are needed if such training is to be effective. \ This 
project is a^Eirst. attempt at creating such, new materials and ap- . 
proaches Vfor the training of students in the art of survey* analysis. 
The project is^exploratory rather than definitive in nature. 

. (Today's student, of survey data is often ait a, loss as\to 
how "to proceed. He is uncertain about how to launch his analysis; 
he is undecided on how to develop his analysis once he has initiated 
it; and he! is unsure about when it is appropriate to 4 raw his analysis 
to a clbse] He is like a novice' chessplayer who merely receives some 
elemental ! instruction concerning the legitimate moves allowed each 
chesspiece \along with a brief introduction concerning the-'bverall 
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objectives of the game. The game played by such a phess novice is i. • 
poor, because (1) he lacks information concerning strategy and tactics 
— i.e., opening game moves designed to develop his pieces, middle 
game moves designed to improve his position and engage his opponent, 
and end game vmoves designed t.o ensure victory -- and, (2) he has no 
understanding of, how to integrate these stages. Like t*he chess 
master, the profession^ survey analyst must thoroughly understand 
the strategy, and tactics for each stage of analysis, anticipating 
several steps in advance. 

New materials and approaches are needed to develop the 
student's insight into the strategy and tactics appropriate to the 
different „ stages' of the research enterprise. This project created 
such novel materials and approaches and then proceeded to try them 
out on social science students . These materials and approaches were 
designed to compensate for the. inadequacies present in, current Ameri- 
can methodology texts -and courses. These new. materials and approaches 
have the following characteristics. They focus on the art of data 
analysis itself rather than on auxiliary processes in the research 
enterprise.^ In the area of data analysis, they concentrate on the 
art of discovery rather than on the logic of verification. They give 
the student ample opportunity to engage in |the actual analysis of 
data rather thdn on being expositional in niature. They hold’ the/ 
student ' s interest and involvement , and they are suitable for use 
fn small as well as .in large colleges and' universities. 



i 






The new materials and approaches we developed were based on 
the assumption that the student best learns data analysis by 1, engaging 
it it. 'Discussion of the principles or exposure to examples of ! 

research are no substitutes for the analysis of data itself, i, It is 1 ^ 
often, however, impractical and inconvenient to plunge novices into 
the confusion of the sea of • raw interview data. , As an alternative, 
we provided the- student with sets of two and threje variable tables 
as theirobasic data input. These tables fall between the extremes 
of raw data contained in interview schedules and polished final 
research reports. * 





These tables were created in three different ways. • Some i- 
were the products of our imagination; somd were taken intact from \. 

. actual survey data ; • and some were 'initially based on survey data , but \ 

; altered for heutistic purposes . Tables created in this last manner 
proved to be the best. Different sets of tables were created in the \ 
following four substantive areas -- voting behavior, race- relations., 
student attitudes and behavior, and social mobility. As a result 
of our experiences in using these materials in methodology courses, 

. ;we «e of the opinion that in the future it would be wiser to restrict, 
the use of such material to a single substantive area. These tables 
were extens ively used in our . training games and formats that will be •' 
discussed.. These games and formats constitute the novel materials 
and approaches developed by us. 
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We had hoped to enlist the voluntary help of colleagues 
in evaluating the effectiveness of the materials we had developed. 

We distributed copies of our materials , along with evaluating tests, 
to a number of colleagues in sociology or political science depart- 
ments at different^ institutions where -they were teaching survey re- 
search methods courses. Unfortunately, we have not received the feed 
back from these colleagues. As a result*, this report is largely limit 
ed to- our own impressions based on our -experiences with the materials 
in four graduate level methodology courses. This approach, although , 
inappropriate for rigorous testing, h3s value in an exploratory pro- 
ject, such as ours. IfUllous for. flexibility which facilitates the ( 
discarding of approaches that fail to fulfill promise. It stimulates ! 
experimentation, with the prepared materials, and it promotes spon- 
taneous development of new materials suggested by those experiences . 






I 

i 



* . i 



! All students were initially exposed to a brief instruction . 
on the rudiments of survey analysis;. The content of this instructipn 
was .pared down to the bare essentials, and the format of presentation 
was changed from an abstract one to a problem- oriented approach. The 
students did not master the art of survey analysis in the six (around 
six) meetings devoted to this basic instruction. Nevertheless, they 
developed a preliminary understanding of the logic of survey analysis 
\ along with an appreciation of its value in illuminating problems 
existing in the real world. In addition, they acquired a basic vocab- 
ulary that facilitated! their reading of methodology articles. 



After being introduced. to the ABC's of survey analysis, the 
students were exposed to a series of games, which were developed by 
us to initiate students 'into the art of data analysis. They utilized 
the sets of tables described earlier, as well as scripts containing 
story lines pertaining to these tables. In the first game, the class 
was divided into several groups, and each group wjas split into two 
teams. Each group was given a set of tables and a script,. and the 
two teams of each group played against each other. One team tried to 
demonstrate that the data presented in the tables supported the story 
line developed in the script; while the other team attempted to show 
that the story line was unsubstantiated, by the data. The two teams 
gave presentations before the entire class j which judged on which 
team made the better presentation. This game was well received by the 
students . 



Our experiences suggested several ways in which the amount 
of game time could be reduced. By increasing the size of each group, 
we reduced the time- needed for classroom presentations. By insisting 



that discussion among, members of each- team. as well as debates between 

i » 



teams , take place outside of ‘the- classroom, valuable 'class t ime t?as 



saved. Finally, presentations were made oniy to half, rather than 
.the entirer class; therefore, two separate presentations could' occur 
Simultaneously; The elaborate point scoring scheme designed to foster 
competition between teams was replaced by a round -robin format. This 
format retained the competitive element but eliminated the confusion 
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emerging from the involved point-scoring system. This game could be 
improved by giving each team an option on whether or not it wishes 
to defend the tables. ' — — 



After a teem had emerged "victorious" from the round-robin 
contest,* Its members played other data- analysis games among themselves. 
One such game was similar to the game discussed above*-- i.e,, match- 
ing sets of tables to scripts -- except that in this case, competition 
cook place among members of a single team rather than between teams; 
^mother game involved .giving each team additional tables at intervals 
and having them indicate ways in which the additional tables altered, 
the previous fit between 'data ?nd script. Still another game called 
upon students to create scripts based- on their analysis of the tables. 
The students found it more difficult to spin stories from^data theft'*' ' ... 
to search through a set of tables in order to substantiate a prepared' 
story,. Another game sought to sharpen the students' awareness of the', 
relationship between tables, analyses, and interpretations was one in 1 
which one student received table-cards and proceeded to discard them „ 
’one at a time. A second student, supplied with analysis-cards, tried 
to match each table-card with an analysis^card. A third student, 
given only interpretation-cards , attempted to. find an appropriate \ 

interpretation-card; It is our judgment that thi/s game would be more \ 
effective if it were us ^ 4 . earlier, rather than late, in the training ^ \ 

period. The final game was our domino game. . Students received cards* 
representing the relationship between different pairs of variables. 

The complete set of cards relat'ed to a single spciological phenomenon. 
Although this game was initially designed for a group of players, we 
decided to restrict, play to only two players at a time. .One player 
would throw out a bard; the other player tried to match one of his 
cards teethe discarded cards. Eventually, a structure of cards would 
be formed, like a pattern of dominoes ^ with the players giving verbal ' 
interpretations of the connected tables*. 



After all the student’s had had opportunity to play these 
games, they 4 were exposed to training formats. These formats were 
. designed to further sharpen the analytical skills, and they demon- 
strated to the students how .tables were woven together into a final 
research report. This demonstration proceeded step by step through 
several stages — (1) the .polished research’ report , (2) an unveiling 
of the actual analytical path pursued by the author of the report, 

(3) a discussion of alternative paths that coUid have been pursued by 
the author, (4) an opportunity for the student to discovei for him- 
self some other alternative paths of analysis, and, finally, (5) giving 
the student a comprehensive set of tables to analyze and weave into a 
report.- One format began with the final report and traced the stages 
backward, ending with the raw 4ata ; while a second format began with 
the data and culminated in the report. In practice, neither format 
was deemed superior to the other. Some formats -were^created from 
actual research reports ;< some, from our imagination: and £ome began 
as actual reports" but were altered for hepristic purposes. ‘Formats * . 
created in this latter fashion worked out best in classroom use. 



The games and formats .were designed to familiarise the 
student with the strategy and tactics ’of, survey analysis. We initial- 
ly chose to ignore the other phases involved in the research enter- 
prise. In practice, however^. we decided to sensitize tne students to 
the effect of questionnaire construction on the subhe^uent analysis- 
of data. In order to do this, several exercises were developed. . 
Students were asked to spin off stories from sets of tables derived \ 
from questionnaires which lacked crucial questions. < The students 
were required to write up additional questions that would improve. the 
analysis. In another exercise, the students created a brief question- 
naire of their own. Mock tables were created by us. on the basis/ of 
their questionnaire to highlight the abs'ence of certain questions: * 

Finally, the students were exposed to the different re'sults obtained 
when- si ight^ rewording of questions, takes place. In total, these exer- 
cises mide/the students more aware of the problems of questionnaire 
construction and its relationship to data analysis.^ * * 

12. Conclusions ' . 

By and large, the students were enthusiastic about the 
games. Our flexible approach stemming from the exploratory nature, 
of this project encouraged us to make numerous changes in these games 
for improving their <«rS3e. These games, as modified, were indeed effect- 
ive in introducing students to the art of survey analysis. Students 
seemed to understand the ways in which survey {lata could be utilized 
for the purposes of confirming, or rejecting hunches, hypotheses, and 
theories. They also, appeared to fathom the w&ys in which combina- 
tions of tables can form the foundation of complex explanations and 
interpretations of sociological phenomena. Rather than being intim- 
idated by the tables, most of the Students appeared at ease with the 
data. Finally, the analysis that several students carried out on 
actual computer processed survey data was of a much higher calibre 
than that carried out by students exposed only to conventional train- 
ing ; in survey analysis. Finding tables to corroborate a given story 
line presented no problems; however, the students still had difficulty 
developing a stp.ry line from tables. It had been hoped that the new 
formats which we developed would have remedied this weakness. 



